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DETAILED ACTION 

This action is responsive to communications: Amendment, filed on 1/22/07. 

Claims 30-49, and 54-57 are pending in this application. Claims 30, 34, 38, 42, and 46 
are independent claims. In the Amendment, filed on 1/22/07, claims 1-29, and 50-53 are 
canceled, claims 30, 33, 34, 37, 38, 41, 42, 45, 46, 49, and 54-57 were amended. 

Specification 

The incorporation of essential material in the specification by reference to an unpublished 
U.S. application, foreign application or patent, or to a publication is improper. Applicant is 
required to amend the disclosure to include the material incorporated by reference, if the material 
is relied upon to overcome any objection, rejection, or other requirement imposed by the Office. 
The amendment must be accompanied by a statement executed by the applicant, or a practitioner 
representing the applicant, stating that the material being inserted is the material previously 
incorporated by reference and that the amendment contains no new matter. 37 CFR 1 .57(f). 

Applicant did not make reference to foreign application 35688/1999 in its specification 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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Claims 30-49, and 54-57 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Harada US Patent Application Publication 20070157225 further in view Liou US Patent 
6,278,446. 

As per claim 30, Harada teaches an apparatus for processing multimedia data, the 
apparatus comprising: 

A memory; (see Harada figure 8, item 64; Data totalizing apparatus is a memory) and 
A processor configured to perform operation comprising: 

Receiving segment information about an audio-visual program, the segment information 
identifying a plurality of audio-visual segments in the audio-visual program, (figure 16B, item 
Information Broadcast; paragraphs 0104-0105) wherein each audio-visual segment is defined by 
a temporal position in a multimedia stream of the audio-visual segment is defined by a temporal 
position in a multimedia stream of the audio-visual program and represents a continuous 
temporal content portion in the audio-visual program; (figure 16c, item display screen; 
paragraphs 0104-0105) 

Generating a data structure describing multimedia data and including the segment 
information along with segment group information defining first and second segment groups (see 
figure 4, item 15a; paragraph 0067-0068; content coupon information is a segment information 
and service additional information is another segment information), 

each of the first and second segment group defining a respective set of audio-visual 
segments selected from the plurality of audio-visual segments identified by the segment 
information in the multimedia stream, (see Harada, paragraph 0065; The coupon and the service 
segment information includes in additional information and the content G multimedia stream) 
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wherein said segment group information specifies a respective group type and a 
respective duration for each of said first and second segment groups, the respective group types 
indicating that the first and second segment group represent respective first and second (see 
Harada figure 4, item 15a; paragraph 0067-0068; Content coupon information is corresponds to 
coupon, which correlate its duration with the corresponding program paragraph 0104 and Service 
information is corresponds to service, which its duration is based on timeline schedule 0075) and 

wherein the segment group information includes segment order information defining that 
the segments within the first segment group are ordered relative to each other according to a time 
sequence and the segments within the second segment group are unordered relative to each other 
according to the time sequence; (see Harada paragraph 0069, 0083, 0070 and 0064-65; The 
content coupon information is in one embodiment not order based on time but based on user ID, 
the service information is order based time sequence) and 

Storing the data structure with the segment group information in the memory, (see 
Harada paragraph 0070, the data structures are stored on a local disk) 

However Harada fails to teach a hierarchical data structure and content summaries. 

Liou (US 6,278,446) teaches hierarchical data structure and content summaries, (see 
Liou; col. 3, lines 20-40) 

It would have been obvious to an artisan at the time of the invention to include Liou's 
teaching with Harada in order to allow user to view and manually edit the hierarchical structure 
to make the hierarchical structure substantially useful and meaning to the user. 
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As per claim 31, Harada and Liou teach the apparatus of claim 30. Harada further teacher 
wherein said segment group information includes a level information, (see Harada paragraph 
0043, the coupon content has a discount value information which reduces its level each time the 
coupon is used) 

As per claim 32, Harada and Liou teach the apparatus of claim 3 1 . Harada further 
teaches information defines multiple levels, (see Harada paragraph 0043; the discount level can 
be reduce multiple times) 

As per claim 33, Harada and Liou teach the apparatus of claim 30. Harada further 
teaches wherein each segment contained in the first and second segment groups is defined by a 
respective start time and a respective segment duration, (see Harada paragraph 0075, 0069 and 
0070; The content coupon information is sometimes integrated into time line along with regular 
programming, The service information is also keeps the start time/ mark time and the duration of 
the showing) 

As per claim 34, Harada teaches a method for processing multimedia data, the method 
comprising: 

generating segment information about an audio-visual program, the segment information 
identifying a plurality of audio-visual segments in the audio-visual program, (figure 16B, item 
Information Broadcast; paragraphs 0104-0105) wherein each audio-visual segment is defined by 
a temporal position in a multimedia stream of the audio-visual segment is defined by a temporal 
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position in a multimedia stream of the audio-visual program and represents a continuous 
temporal content portion in the audio-visual program; (figure 16c, item display screen; 
paragraphs 0104-0105) 

generating segment group information for a data structure describing multimedia data; 
(see Harada paragraph 0067, The coupon information and service information which are 
generated by the Center) and 

transmitting said segment group information to a client, (see Harada paragraph 0067, The 
coupon information and service information which are generated by the Center are transmitted to 
the client ) 

wherein said segment group information: 

defines first and second segment groups, (see figure 4, item 15a; paragraph 0067-0068; 
content coupon information is a segment information and service additional information is 
another segment information) each of which includes a respective plurality of segments selected 
from a multimedia stream; (see Harada, paragraph 0065; The coupon and the service segment 
information includes in additional information and the content G multimedia stream) 

specifies a respective group type and a respective duration for each of said first and 
second segment groups; (see Harada figure 4, item 15a; paragraph 0067-0068; Content coupon 
information is corresponds to coupon, which correlate its duration with the corresponding 
program paragraph 0104 and Service information is corresponds to service, which its duration is 
based on timeline schedule 0075) and 
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includes segment order information defining that the segments within the first segment 
group are ordered relative to each other according to a time sequence and the segments within 
the second segment group are unordered relative to each other according to the time sequence, 
(see Harada paragraph 0069, 0083, 0070 and 0064-65; The content coupon information is in one 
embodiment not order based on time but based on user ID, the service information is order based 
time sequence) 

However Harada fails to teach a hierarchical data structure and content summaries. 

Liou (US 6,278,446) teaches hierarchical data structure and content summaries, (see 
Liou; col. 3, lines 20-40) 

It would have been obvious to an artisan at the time of the invention to include Liou's 
teaching with Harada in order to allow user to view and manually edit the hierarchical structure 
to make the hierarchical structure substantially useful and meaning to the user. 

As per claim 35, Harada and Liou teach the method of claim 34. Harada further teaches 
said segment group information includes a level information, (see Harada paragraph 0043, the 
coupon content has a discount value information which reduces its level each time the coupon is 
used) 

As per claim 36, Harada and Liou teach the method of claim 35. Harada further teaches 
information defines multiple levels, (see Harada paragraph 0043; the discount level can be 
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reduce multiple times) 

As per claim 37, Harada and Liou teach the method of claim 34. Harada further teaches 
wherein the segment group information defines a respective start time and a respective segment 
duration for each segment contained in the first and second segment groups, (see Harada 
paragraph 0075, 0069 and 0070; The content coupon information is sometimes integrated into 
time line along with regular programming, The service information is also keeps the start time/ 
mark time and the duration of the showing) 

As per claim 38, Harada teaches an apparatus for processing multimedia data, the 
apparatus comprising: 

a memory; (see Harada figure 8, item 64; Data totalizing apparatus is a memory) and 

a processor configured to perform operations comprising: 

Receiving segment information about an audio-visual program, the segment information 
identifying a plurality of audio-visual segments in the audio-visual program, (figure 16B, item 
Information Broadcast; paragraphs 0104-0105) wherein each audio-visual segment is defined by 
a temporal position in a multimedia stream of the audio-visual segment is defined by a temporal 
position in a multimedia stream of the audio-visual program and represents a continuous 
temporal content portion in the audio-visual program; (figure 16c, item display screen; 
paragraphs 0104-0105) 

receiving segment group information defining first and second segment groups in a data 
structure describing multimedia data, (see figure 4, item 15a; paragraph 0067-0068; content 
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coupon information is a segment information and service additional information is another 
segment information), 

each of the first and second segment groups including a respective plurality of segments 
selected from a multimedia stream, (see Harada, paragraph 0065; The coupon and the service 
segment information includes in additional information and the content G multimedia stream) 

wherein said segment group information specifies a respective group type and a 
respective duration for each of said first and second segment groups (see Harada figure 4, item 
15a; paragraph 0067-0068; Content coupon information is corresponds to coupon, which 
correlate its duration with the corresponding program paragraph 0104 and Service information is 
corresponds to service, which its duration is based on timeline schedule 0075) and 

wherein the segment group information includes segment order information defining that 
the segments within the first segment group are ordered relative to each other according to a time 
sequence and the segments within the second segment group are unordered relative to each other 
according to the time sequence; (see Harada paragraph 0069, 0083, 0070 and 0064-65; The 
content coupon information is in one embodiment not order based on time but based on user ID, 
the service information is order based time sequence) and 

storing the data structure with the segment group information in the memory, (see Harada 
paragraph 0070, the data structures are stored on a local disk) 

However Harada fails to teach a hierarchical data structure and content summaries. 

Liou (US 6,278,446) teaches hierarchical data structure and content summaries, (see 
Liou; col. 3, lines 20-40) 
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It would have been obvious to an artisan at the time of the invention to include Liou's 
teaching with Harada in order to allow user to view and manually edit the hierarchical structure 
to make the hierarchical structure substantially useful and meaning to the user. 

As per claims 39-41, they are of the same scope as claims 31-34. Supra. 

As per claim 42, Harada and Liou teach a method for processing multimedia data, the 
method comprising: 

Receiving segment information about an audio-visual program, the segment information 
identifying a plurality of audio-visual segments in the audio-visual program, (figure 16B, item 
Information Broadcast; paragraphs 0104-0105) wherein each audio-visual segment is defined by 
a temporal position in a multimedia stream of the audio-visual segment is defined by a temporal 
position in a multimedia stream of the audio-visual program and represents a continuous 
temporal content portion in the audio-visual program; (figure 16c, item display screen; 
paragraphs 0104-0105) 

receiving segment group information from a provider; (see Harada paragraph 0067, The 
coupon information and service information which are generated by the Center) and 

storing said received segment group information in a data structure describing 
multimedia data in a client, wherein said segment group information: (see Harada paragraph 
0070, the data structures are stored on a local disk) 

defines first and second segment groups, each of which includes a respective plurality of 
segments selected from a multimedia stream; (see figure 4, item 15a; paragraph 0067-0068; 
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content coupon information is a segment information and service additional information is 
another segment information) 



specifies a respective group type and a respective duration for each of said first and 
second segment groups; (see Harada figure 4, item 15a; paragraph 0067-0068; Content coupon 
information is corresponds to coupon, which correlate its duration with the corresponding 
program paragraph 0104 and Service information is corresponds to service, which its duration is 
based on timeline schedule 0075) and 

includes segment order information defining that the segments within the first segment 
group are ordered relative to each other according to a time sequence and the segments within 
the second segment group are unordered relative to each other according to the time sequence, 
(see Harada paragraph 0069, 0083, 0070 and 0064-65; The content coupon information is in one 
embodiment not order based on time but based on user ID, the service information is order based 
time sequence) 

However Harada fails to teach a hierarchical data structure and content summaries. 

Liou (US 6,278,446) teaches hierarchical data structure and content summaries, (see 
Liou; col. 3, lines 20-40) 

It would have been obvious to an artisan at the time of the invention to include Liou's 
teaching with Harada in order to allow user to view and manually edit the hierarchical structure 
to make the hierarchical structure substantially useful and meaning to the user. 
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As per claims 43-45, they are of the same scope as claims 31-34. Supra. 

As per claim 46, Harada teaches a storage medium storing a data structure describing 
multimedia data, the data structure configured to be processed by multimedia data processing 
apparatus, the stored data structure comprising: 

segment information about an audio-visual program, the segment information identifying 
a plurality of audio-visual segments in the audio-visual program, (figure 16B, item Information 
Broadcast; paragraphs 0104-0105) wherein each audio-visual segment is defined by a temporal 
position in a multimedia stream of the audio-visual segment is defined by a temporal position in 
a multimedia stream of the audio-visual program and represents a continuous temporal content 
portion in the audio-visual program; (figure 16c, item display screen; paragraphs 0104-0105) 

segment group information defining first and second segment groups, each of which 
includes a respective plurality of segments selected from a multimedia stream, (see figure 4, item 
15a; paragraph 0067-0068; content coupon information is a segment information and service 
additional information is another segment information) 

wherein said segment group information specifies a respective group type and a 
respective duration for each of said first and second segment groups, (see Harada figure 4, item 
15a; paragraph 0067-0068; Content coupon information is corresponds to coupon, which 
correlate its duration with the corresponding program paragraph 0104 and Service information is 
corresponds to service, which its duration is based on timeline schedule 0075) and 
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wherein the segment group information includes segment order information defining that 
the segments within the first segment group are ordered relative to each other according to a time 
sequence and the segments within the second segment group are unordered relative to each other 
according to [[a]] the time sequence. . (see Harada paragraph 0069, 0083, 0070 and 0064-65; 
The content coupon information is in one embodiment not order based on time but based on user 
ID, the service information is order based time sequence) 

However Harada fails to teach a hierarchical data structure and content summaries. 

Liou (US 6,278,446) teaches hierarchical data structure and content summaries, (see 
Liou; col. 3, lines 20-40) 

It would have been obvious to an artisan at the time of the invention to include Liou's 
teaching with Harada in order to allow user to view and manually edit the hierarchical structure 
to make the hierarchical structure substantially useful and meaning to the user. 

As per claims 47-49, they are of the same scope as claims 31-34. Supra. 

As per claim 54, Harada and Liou teach the apparatus of claim 30. Harada further 
teaches wherein the respective group type for each of the first and second segment groups 
specifies that the first and second segment groups are related to at least two objects represented 
in the content of the multimedia stream, (see Harada figure 4, item 15a; paragraph 0067-0068; 
Coupon segment is related to target audience and Service is related to target client) 
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As per claim 55, Harada and Liou teach the apparatus of claim 54. Harada further 
teaches wherein the first segment group includes segments representing time ordered relation 
changes between the at least two objects. . (see Harada paragraph 0075, 0069 and 0070; The 
content coupon information is sometimes integrated into time line along with regular 
programming, The service information is also keeps the start time/ mark time and the duration of 
the showing) 

As per claim 56, Harada and Liou teach the apparatus of claim 30. Harada further teaches 
wherein the segment group information indicates that the first segment group includes segments 
that represent highlights from the multimedia stream, (see Harada paragraph 0075, 0069 and 
0070; The service information is also keeps the start time/ mark time and the duration of the 
showing, the mark time includes highlights) 

As per claim 57, Harada and Liou teaches the apparatus of claim 54. Harada further 
wherein the segments within the second segment group include segments representing constant 
relations between the at least two objects, (see Harada paragraph 0075, 0069 and 0070; Hit target 
user information is correlated to Id of a service to be discounted) 

Response to Argument 

Applicant's arguments with respect to claims 30-49, and 54-57 have been considered but 
are deemed to be moot in view of the new grounds of rejection. 
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Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to SIMON KE whose telephone number is (571)272-4062. The 
examiner can normally be reached on M-Th and Alternate Fridays 8:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David A. Wiley can be reached on (571) 272-3923. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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